System of Language Learning with Augmented Reality

ABSTRACT

Education curricula materials include encoded indicia such as a QR code that contains information related to identifying requested augmented reality image data from a server over a network. By scanning the QR code, a computer uses its decoding software to create a data set for transmitting to the server. The data set may include an identifier for selected augmented reality image data associated with the user&#39;s curriculum, information about the curriculum at issue, the academic level of the user, and any other data necessary to ensure that the most appropriate augmented reality image data is transmitted back to the computer. The server transmits comprehensive augmented reality image data back to the computer for viewing on a computerized display accessible by a student. Part of the content may include an interactive pedagogical agent that helps the student with a part of the instruction related to a portion of the curriculum.

CROSS REFERENCE TO RELATED APPLICATIONS

This application incorporates by reference and claims priority to U.S.Provisional Patent Application Ser. No. 62/822,125, entitled More thanWords: Augmented Reality Vocabulary Instruction for English LanguageLearners, filed on Mar. 22, 2019.

FIELD OF THE INVENTION

The system and method disclosed herein relate to using augmented realitycomputer systems to provide interactive experiences for languagelearners that reinforce vocabulary and reading skills.

BACKGROUND

Modern educational systems have developed computerized machinery andassociated methods by which individuals can perform numerous interactivelearning techniques electronically. Online and virtual instruction arecommon tools in the modern school environment, yet even theseadvancements fail to address deficits that students face when they havebeen speaking a first or native language for long periods of time andthen have to learn a new language that is fully immersed into theirclassroom situation.

Nearly one third of U.S. children ages 0-8 have one or more parents whospeak a language other than English in the home (Park, Zong & Batalova,2018). Children who speak a language other than English at homepersistently lag behind monolingual English-speaking peers in readingand math in 4th, 8th and 10th grades (NAEP, 2010; NOES, 2015), yet themajority were born and schooled in the U.S. (Park, Zong & Batalova,2018). Early achievement gaps have long-term consequences forindividuals and society.

Insufficient vocabulary knowledge is a well-documented contributor toEnglish language learners (ELLs) having academic difficulties (August &Shanahan, 2006; Carlo et al., 2009; NAEP, 2010). Monolingual Englishspeakers enter school with an estimated 5,000 to 7,000 words, while ELLsbegin with 3,000 to 6,000 English words, or, in some cases, as few aszero (Carlo et al., 2009). As schooling progresses, vocabulary demandsescalate and school texts become progressively complex; ELLs who lagbehind in vocabulary are increasingly unable to comprehend grade-levelmaterial (Carlo et al., 2009; Durgunoglu & Goldenberg, 2010).Insufficient vocabulary knowledge constrains reading comprehension andstymies the process of learning new words from reading (Carlo et al.,2009). As such, there is pressing need for pedagogical innovations inELL vocabulary instruction to help narrow current achievement gaps.

Linguistic diversity is predicted to increase with shifting migrationtrends (decreasing from Mexico, increasing from Africa, Asia; Park, Zong& Batalova, 2018). Nearly one third of U.S. children ages 0-8 have oneor more parents who speak a language other than English in the home(Park, Zong & Batalova, 2018). Children who speak a language other thanEnglish at home persistently lag behind monolingual English-speakingpeers in reading and math in 4th, 8th and 10th grades (NAEP, 2010; NCES,2015), yet the majority were born and schooled in the U.S. (Park, Zong &Batalova, 2018). Early achievement gaps have long-term consequences forindividuals and society.

Research indicates that children need 12 or more exposures to a novelword, ideally accompanied by robust supporting information, to acquirefull use of the word in their vocabulary (Biemiller and Boote (2006);McKeown, Beck, Omanson, and Pople, (1985) note in their work that thisnumber can be prohibitively high for ELLs exposed to English primarilyonly during school.

Augmented Reality. Augmented Reality (AR) refers to a viewing system(commonly, a mobile device camera lens) that superimposes virtual 3Dobjects over the viewer's view of the surrounding world, creating theillusion of virtual objects in the viewer's environment. AR rendered“objects . . . appear to coexist in the same space as the real world,”creating an immersive, real-time, interactive experience that enhancesthe user's understanding of the world (Azuma et al., 2001, p. 34).“Target-based AR” allows users to point a mobile device camera lens at adesignated “target,” a specific image such as a QR code (square barcodesreadable by smartphones), to view aligned virtual content (e.g. audio,video, graphics; Dunleavy, 2014).

AR is uniquely situated to enhance learning by offering many forms ofmultimedia content, possibilities for differentiation, and easyalignment with existing materials, such as books. AR createsparticipatory experiences in which learners construct knowledge,interact with real and virtual others, and interpret meaning;characteristics emphasized in theoretical learning models (i.e. situatedlearning theory, constructivist learning theory; Dunleavy & Dede, 2014).An emerging body of educational research among children indicates thatAR has the ability to increase motivation and enjoyment (Cascales etal., 2013; Lu & Liu, 2015; Di Serio et al., 2012) and learning outcomes(Chiang et al., 2014; Yoon & Wang, 2014). Additionally, AR'slearner-driven features increase feelings of ownership and self-learning(Ferrer-Torregrosa et al., 2015). AR has been demonstrated to supportearly literacy development (Silva et al., 2013) and has been used asearly as preschool (Han et al., 2015; Cascales et al., 2013). AR canalso support the delivery of content in a child's first language whenteachers lack the language proficiency to do so.

Augmented reality systems offer the use of a pedagogical agent (PA), ananimated character who supports learning by engaging with the user, thatcan increase motivation and learning (Park, 2015). PAs deliverinstructions, encouragement, and demonstrate concepts (Park, 2015).Research indicates that PAs improve motivation, attitude (Atkinson,2002, Baylor & Ryu, 2003), and support e-book reading comprehension forchildren (Karemaker et al., 2017).

As noted, augmented reality offers many features relevant to languagelearning, yet comparatively little research has examined augmentedreality for vocabulary. According to Clark & Paivio's (1991) dual codingtheory, images and words are processed separately by memory; therefore,simultaneous utilization of both results in greater retention. Augmentedreality goes beyond traditional instruction by presenting digital media,such as spoken definitions, expanded descriptions, sounds, multiplepictures, 3D animations, videos, and/or print. Among children, readinge-books with dictionaries containing multimedia meanings and audio cuessupport word recognition skills better than “flat” dictionaries(Karemaker et al., 2017). Developing AR literature on English as foreignlanguage indicates that AR increases vocabulary learning among children(Barreira et al., 2012), decreases cognitive load and improves attitudeamong children (Kü

cük, Yilmaz, & Göktas, 2014), and improves comprehension among adults(Liu & Tsai, 2013).

Augmented Reality for ELLs. AR is particularly suited to addressspecific challenges intrinsic to the ELL context, including: providingrobust support in the first language (L1), ensuring novel words areencountered a sufficient number of times, with optimal semantic support,and instruction differentiation for increasingly diverse classrooms.ELLs learn new English words best when provided supporting materials inthe first language (Carlo et al., 2009). Providing first languagecontent in many classrooms can be challenging, however, as teachers arerarely proficient in all students' first languages.

A need exists for new computerized systems that take advantage ofaugmented reality platforms to increase efficiencies when teachingstudents that are fully immersed into a new language at school.

BRIEF SUMMARY OF THE EMBODIMENTS OF THE DISCLOSURE

In one embodiment, a computerized system of teaching utilizes curriculummaterials having encoded indicia thereon. The system includes a computerhaving computerized memory and a processor executing imaging softwareand decoding software in the memory. A camera includes an image readerassembly configured to generate pixel data from the encoded indicia on arespective curriculum material, wherein the imaging software processesthe pixel data to generate digital pixel data and stores the digitalpixel data in the memory, wherein the processor executes decodingsoftware by receiving the digital pixel data from the memory andgenerates a data set from the encoded indicia. A transceiver is inbi-directional communication with the computer and a server on anetwork, and the server has access to augmented reality image data. Thetransceiver transmits the data set to the server over the network, andthe server is configured to receive the data set and transmit, to thecomputer, selected augmented reality image data that corresponds to thedata set. The computer uses the processor to show the augmented realityimage data on a display.

In another embodiment, a computer program product includes a set ofcomputer instructions stored on non-transitory computer readable mediahoused in a computer, and the instructions are configured to be executedby a processor to implement a decoding process in regard to an image ofan encoded indicia on a curriculum material. The computer instructionsinclude an imaging module that (i) activates a camera in the computer tocapture an image of the encoded indicia, (ii) generates a set of digitalpixel data corresponding to said encoded indicia, and (iii) stores saiddigital pixel data in addressed memory locations in the computer. Adecoding module processes at least a portion of the digital pixel datato generate a data set corresponding to a respective portion of digitalpixel data stored in memory. A transmission module is configured toexecute a data transmission via a network in communication with thecomputer, the data transmission sending the data set to a remote serverthat identifies corresponding augmented reality image data requested bythe data set. A second set of computer instructions stored on thenon-transitory computer readable media and executable by the processorreceive via the network the augmented reality image data from the remoteserver and display the augmented reality image data on the computer.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a schematic view of an exemplary computer system operating ona network as described herein.

FIG. 2A is a front side view of curriculum material in the form of afirst vocabulary flash card having an encoded indicia thereon asdisclosed herein.

FIG. 2B is a rear side view of the first vocabulary flash card of FIG.2A.

FIG. 2C is an environmental view of the first vocabulary flash card ofFIG. 2A that has been used to activate an augmented reality view of thefirst vocabulary flash card on a computer as disclosed herein.

FIG. 3A is a front side view of curriculum material in the form of asecond vocabulary flash card having an encoded indicia thereon asdisclosed herein.

FIG. 3B is a rear side view of the second vocabulary flash card of FIG.3A.

FIG. 3C is an environmental view of the second vocabulary flash card ofFIG. 3A that has been used to activate an augmented reality view of thesecond vocabulary flash card on a computer as disclosed herein.

FIG. 4A is a front side view of curriculum material in the form of athird vocabulary flash card having an encoded indicia thereon asdisclosed herein.

FIG. 4B is a rear side view of the third vocabulary flash card of FIG.4A.

FIG. 4C is an environmental view of the third vocabulary flash card ofFIG. 4A that has been used to activate an augmented reality view of theflash card on a computer display as disclosed herein.

FIG. 4D is an environmental view of the third vocabulary flash card ofFIG. 4A that has been used to activate a virtual projection of anaugmented reality three dimensional pedagogical agent thereon asdisclosed herein.

DETAILED DESCRIPTION

The terms of this detailed description are intended to have theirbroadest plain meaning. For example, “software” includes, withoutlimitation, instructions, whether in source code or compiled versionsand whether activated by processors, firmware, or other hardware so longas the computerized electronics have the physical parameters that enableactions taken in accordance with the logic of the instructions.“Computer readable media” that implement “computerized methods” alsoencompass tangible hardware such as data storage media, or communicationmedia including any medium that facilitates sharing data within asystem. Computer-readable media generally include tangiblecomputer-readable storage media which are non-transitory in their use ofelectronic signals or waves that are useful in connection with themedia. Terms such as non-transitory media are used to recite variouscomputer readable media without relying upon the transitory nature ofphysical signals. Computer-readable storage media can comprise RAM, ROM,EEPROM, flash memory, or any other medium that can be used to storedesired program code in the form of instructions or data structures andthat can be accessed and used by a computer.

The term “network” encompasses all forms of data communicationstraversing an infrastructure designed to share electronic data betweenat least two devices. Networks often encompass user devices, bothwireless and wired, as well as gateways, servers, wireless transceivers,and similar hardware that establish a communications protocol allowingnodes on a network to share information of all kinds. One non-limitingexample of a network is the commonly used internet infrastructure.

FIG. 1 outlines a brief synopsis of a system and method by which Thetechnology for the system, apparatus, and method of this disclosureincludes computerized devices 100, such as smart phones, tablets,personal computers and the like, which are equipped with cameras andimaging devices 101 capable of transmitting receiving and transmittingimage data. The term “image data” is given its broadest meaning in thisdisclosure to include all data associated with augmented realitysystems, including but not limited to, video data, audio data, andassociated telecommunications data necessary to accomplish the augmentedreality images of this disclosure. In one non-limiting embodiment, animaging device 101 may be a camera connected to a smart device, such asa smart phone or other portable communications computer. The computerused in this disclosure includes all of the technology and hardware thatwould be expected in modern computing and certainly sufficient toprocess image data. The computer of FIG. 1 includes a processor 108,memory 110, a clock 114, a power circuit 118, and communicationshardware 125 that is connected to a network 140. A server 850 is alsoconnected to the network and provides augmented reality image data backto the computer.

To add to the efficiency of learning systems described herein, onesystem herein incorporates the option of using encoded indicia that cansecurely incorporate numerous kinds of identifying data about avocabulary word that is a planned part of a student's readingassignment. Encoded indicia, as used herein, include but are not limitedto bar codes, QR codes, or any kind of printed symbols that relay datain a format that requires decoding to retrieve underlying data from theindicia. In one embodiment, the encoded indicia are printed directlyonto a student's curriculum materials, such as vocabulary flash cardsthat are provided with reading assignments. The system and method alsoencompass the option of utilizing printed symbols, letters, numbersand/or other images that are not encoded but are simply read ortranslated into a digital format by appropriate software on a computer.

The encoded indicia of this disclosure are examples of computerizedtools allowing students and teachers to access augmented realityinstruction for educational purposes. Children who speak a languageother than English at home often enter school with lower Englishproficiency, which hinders their ability to meet grade-level readingexpectations and ultimately puts them at risk for academic difficulties(Carlo et al., 2009; NAEP, 2010; NCES, 2015). Augmented Reality (AR), asystem that creates the illusion of virtual objects in the viewer's realsurrounding environment by superimposing multimedia over the real view,offers unique possibilities for delivering engaging, differentiated,robust vocabulary instruction for English Language Learners (ELLs). Thisdisclosure shows an AR prototype to accompany a known and existingcurriculum, such as but not limited to the Developing Talkers (DT)curriculum, a research-based program for increasing students' academiclanguage skills using repeated shared book reading (SBR) and vocabularypicture cards with explicit instruction. The proposed AR applicationadds robust supporting multimedia content to vocabulary picture cardinstruction. Using iterative design methodology, the current studyemploys a cyclical process of creation, evaluation, and improvement withchild participants to demonstrate proof-of-concept, cost, and childacceptance. AR is potentially transformative for vocabulary instruction;the proposed study would represent a major contribution to the field andestablish new cross-disciplinary and cross- institutional relationships.

AR, in conjunction with curriculum (flashcards, books), can increaseword exposure and provide rich support for challenging, abstract, orcomplex concept words. Target-based AR is adaptable and offersopportunities for fine-grained differentiation. Materials cansimultaneously offer multiple “settings” through targets that presentmaterial differentiated by English skill, first language, learninggoals, setting (home, school), and/or users (solo, with parent, withteacher). Adding AR to existing pedagogical materials creates homelearning modes that do not rely on parent English proficiency. Teacherscan direct children to appropriate targets for robust, individualizedinstruction from a PA at school or home, ameliorating pressures relatedto limited class time and large class sizes.

In one embodiment, a computerized system of teaching utilizes curriculummaterials having encoded indicia thereon. The system includes a computerhaving computerized memory and a processor executing imaging softwareand decoding software in the memory. A camera includes an image readerassembly configured to generate pixel data from the encoded indicia on arespective curriculum material, wherein the imaging software processesthe pixel data to generate digital pixel data and stores the digitalpixel data in the memory, wherein the processor executes decodingsoftware by receiving the digital pixel data from the memory andgenerates a data set from the encoded indicia. A transceiver is inbi-directional communication with the computer and a server on anetwork, and the server has access to augmented reality image data. Thetransceiver transmits the data set to the server over the network, andthe server is configured to receive the data set and transmit, to thecomputer, selected augmented reality image data that corresponds to thedata set. The computer uses the processor to show the augmented realityimage data on a display.

The computerized system of this disclosure may be used with a projectorto show the augmented reality image data, such as an active pedagogicalcharacter, in three dimensional space around the user, such as the PAstanding on a flash card in FIG. 4D. In all cases, the augmented realityimage data may include still images, video data, audio data and allnecessary telecommunications data for a synchronized presentation ofaugmented reality instruction on the user computer or projector. Theuser, such as a student, will see the augmented reality instructionmaterials incorporated like a video or cartoon playing atop and inconjunction with the student's real environment. Computers used inconjunction with this disclosure may be equipped with interactiveequipment that allows the student to participate in exercises within anaugmented reality framework. For example, the student may use agraphical user interface, a mouse, a keyboard, and/or a microphone toparticipate in the instruction presented by a pedagogical agent in theaugmented reality view.

In certain non-limiting embodiments, the concepts herein are used inconjunction with other written materials, whether accessed by hardcopies or software. The examples of FIGS. 2-4 illustrate certain ways toaccess and use the augmented reality system with hard copy writtenmaterials, but the examples are not limiting. In the Figures, a certaincomponent of the curriculum materials, such as vocabulary word flashcards 200A, 300A, 400A bear an encoded indicia 210, 310, 410 on at leastone side of the vocabulary word flash card. An imaging device 101connected to or part of a computer 100 can be used to scan the encodedindicia to implement the augmented reality system described herein. Onegoal is to supplement information printed on the flash card, such as thephoto image 205A, 305A, 405A on a first side of the flash cards and theinstructional text 205B, 305B, 405B located on a second side of theflash cards. The flash cards, therefore provide written instructions,photo support, and access to augmented reality instruction through theencoded indicia 210, 310, 410.

As briefly described above and illustrated in FIGS. 2, 3, and 4, acamera or other imaging device 101 used by a student, teacher or parent,includes an image reader assembly configured to generate pixel data fromthe encoded indicia 210, 310, 410 on a respective curriculum material,such as one of the vocabulary word flash cards 200, 300, 400. Theimaging software processes the pixel data to generate digital pixel dataand stores the digital pixel data in the memory 110, wherein theprocessor executes decoding software by receiving the digital pixel datafrom the memory and generates a data set from the encoded indicia 210,310, 410. A transceiver is in bi-directional communication with thecomputer 100 and a server 150 on a network 140, and the server hasaccess to augmented reality image data. The augmented reality image datamay be stored in a database or in the cloud as provided by numerouscontent providers. The transceiver transmits the data set to the server150 over the network 140, and the server is configured to receive thedata set and transmit, to the computer, selected augmented reality imagedata that corresponds to the data set. The computer 100 uses theprocessor to show the augmented reality image data on a display 220,320, 420.

In certain non-limiting embodiments, the encoded indicia is a QR codethat contains information related to identifying requested augmentedreality image data from the server 150 over the network 140. By scanningthe QR code, the computer 100 uses its decoding software to create adata set for transmitting to the server. The data set may include anidentifier for selected augmented reality image data associated with theuser's curriculum, information about the curriculum at issue, theacademic level of the user, and any other data necessary to ensure thatthe most appropriate augmented reality image data is transmitted back tothe computer.

The server 150 transmits comprehensive augmented reality image data backto the computer 100 for viewing on a display accessible by a student. Asnoted above, part of the content may include an interactive pedagogicalagent 270, 370, 470 that helps the student with a part of theinstruction related to a portion of the curriculum.

In another embodiment, a computer program product includes a set ofcomputer instructions stored on non-transitory computer readable media(e.g., memory 110) housed in a computer 100, and the instructions areconfigured to be executed by a processor to implement a decoding processin regard to an image of an encoded indicia 210, 310, 410 on acurriculum material 200, 300, 400. The computer instructions include animaging module that (i) activates a camera 101 in the computer 100 tocapture an image of the encoded indicia, (ii) generates a set of digitalpixel data corresponding to said encoded indicia, and (iii) stores saiddigital pixel data in addressed memory locations in the computer. Adecoding module processes at least a portion of the digital pixel datato generate a data set corresponding to a respective portion of digitalpixel data stored in memory. A transmission module 125 is configured toexecute a data transmission via a network 140 in communication with thecomputer 100, the data transmission sending the data set to a remoteserver 150 that identifies corresponding augmented reality image datarequested by the data set. A second set of computer instructions storedon the non-transitory computer readable media and executable by theprocessor 108 of the computer 100 receive via the network 140 theaugmented reality image data from the remote server and display theaugmented reality image data on the computer.

These technical aspects of the disclosure have been implemented in thecase study that follows.

Case Study

The current study proposed to develop and test a target-based augmentedreality

(AR) prototype to augment existing vocabulary cards that form part of anearly childhood oral language and vocabulary instruction curriculumsupplement known as Developing Talkers (DT). DT uses research-basedpractices for increasing students' academic language skills usingrepeated shared book reading (SBR) experiences as the cornerstone. TheSBR routine includes vocabulary picture cards with teaching instructionsfor use before and after shared book reading, elaborations duringreading, and guiding questions that became increasingly challenging.Augmented Reality (AR) multimedia content will align with existingcurriculum and present users with: 1) Pedagogical Agent (PA) deliveredaudio of rigorously developed, child-friendly definitions and scriptedexplanations designed to be used by children with help from teachers andpossibly parents before and after reading corresponding storybooks; 2)pictures and animations. Support explanations will be delivered by PAsScout (a squirrel) and Luz (a Spanish speaking ladybug), charactersalready integrated into the Developing Talkers (DT) curriculum. AR willbe added by placing QR code stickers on vocabulary cards. The proposedstudy has 3 aims: 1) develop an AR prototype to supplement the DTcurriculum and to allow greater flexibility of use by children andadults; 2) demonstrate proof-of-concept, cost, and child acceptance; 3)conduct a learning experience analysis.

Project Activities and Outcomes.

The study used iterative design methodology, a cyclical process ofcreation, evaluation, and improvement. This method is commonly used fornew technology applications and is effective for facilitating designimprovement. First, target-based AR for a chosen curriculum was created,then it was evaluated in research sessions with child participants.Based on session findings, the app has been improved then evaluatedagain. The AR app underwent approximately 5 evaluation rounds with 2participants per round.

Participants. Ten typically developing Spanish-speaking ELLs betweenages 4-6 were recruited via convenience sampling, flyers, and emailListServs.

Audio recordings. Supporting audio material will be recorded in asound-proof testing and recording booth, that has on prior occasionsbeen made available to us by the Department of Communication Science andDisorders, and edited using Adobe Audition software.

AR App. AR materials will be created using GoAnimate software, HP Revealapp and HP Reveal studio. HP Reveal is a free service for creatingAR-based learning activities, and offers easy-to-use interfaces forimmersive AR experiences via iOS/Android mobile devices.

QR Targets. QR Code-based triggers will be created via QR code website.QR targets will be printed on sticker paper and placed on flashcards.

Tablets. Participants will access AR materials using Samsung Tabletssupplied by Dr. Park.

Location. Research sessions will take place at the College of Behavioraland Community Sciences clinic research observation rooms, 12×12 privaterooms equipped with video and audio recording, observation window, tableand chairs.

Sessions. The Augmented Reality (AR) app will undergo a total of 5evaluation rounds, each consisting of 2 individual participant sessions.The initial two rounds of evaluation will be with researcher, rounds 3-5will include parent. During the session, participant will be presentedwith cards and Samsung tablet. The researcher will present theparticipant with 5 AR vocabulary cards, demonstrate tablet use and ARcapabilities, then spend 10 minutes reviewing target cards. Inevaluation rounds 3-5, parents present and review cards for 10 minutes.After 10 minutes of card work, child and researcher/parent will read analigned storybook containing target words, then again review AR cardscontaining post-reading support for target words. Finally, researcherwill debrief participant and parents, asking about enjoyment and memoryfor cards, and administer target word multiple choice test (see Childacademic vocabulary measure).

Curriculum. Lessons feature trade books (half narrative and halfinformational genres) that are read in whole or small groups and aresupported with vocabulary instruction and inferential language andcomprehension instruction. Each book includes 9-12 target vocabularywords that are interactively taught with the aid of picture vocabularycards before and after reading and verbal elaborations during reading.To select target words, curriculum focused on sophisticated wordsaccording to Beck and colleagues' criteria (2013) and sought wordsmastered at grade 4 or beyond in past research (Dale & O'Rourke, 1981)to ensure these were unfamiliar words worth teaching. The majority ofwords selected were sophisticated words (77.98% “Tier 2”; Beck et al.,2013) and estimated to be mastered at grades 4 or 6 (Dale & O'Rourke).Taught words include all parts of speech: verbs, nouns, modifiers. Eachbook also includes a guiding question that is presented prior to readingthe book and multiple literal and inferential questions on stickersplaced in the text at the point of use.

Child academic vocabulary measure. Receptive understanding of the 5 ARinstruction words and 5 exposure-only words that appear in curriculumtext but receive no explicit instruction. This measure is presented on atouchscreen, tablet computer. After two practice items, children listento audio recorded instructions that tell them to “Point to the bestpicture for . . . (target word)” as they are presented with threephotographs—one of which demonstrates the target word and two foils.Foils are selected to ensure similarity of image features unrelated tothe concept tested. For example, foils maintain similarity in the age,number of subjects and of relative size, orientation and saliency ofobjects across the foils.

Learning experience analysis. Analysis will evaluate: 1) cognitiveexperience (i.e. learning gains as measured by post-test, reportedperceived task difficulty); 2) emotional experience (i.e. situationalinterest, self-reported positive and/or negative emotions); 3) behaviorexperience (i.e. observed total time taken to complete each learningactivity, behavioral sequence of completing a learning activity such asviewing activities, questions asked to teacher/parents during activity,AR card interaction including flipping, sorting, etc.).

Outcomes. The proposed project will develop an AR prototype that couldsupplement the Developing Talkers curriculum to demonstrateproof-of-concept, cost, and child acceptance through an iterative designmethodology.

The following scripts were considered for the augmented reality imagedata viewed by subjects who interacted with the pedagogical agent (PA)as follows.

Script 1 (see FIG. 2)

-   PA: Swoop-   Watson the Sloth: Sw-oop, swooop.-   PA: Swoop starts with S-   “S” is made salient-   PA: Swoop-   PA: These airplanes were flying yesterday. Yesterday, they swooped    through the air.-   “Swoop” changes to “Swooped” when PA says “Swooped”-   PA: When something swoops, it moves quickly through the air toward    the ground.-   “Swooped” change to “Swoops” when PA says “Swoops”-   Chime noise-   “Swoops” changes to “Bajar en picada”-   PA: Swoop quiere decir bajar en picada. Cuando algo baja en picada    se mueve rápido por el aire hacia la tierra.-   “Bajar en picada”changes to “Swoop”-   PA: Look at me swoop my hand. Let's act out swooping, you move your    hand toward the ground-   PA swoops hand.-   Script 2 (See FIG. 3)-   PA: Brave-   Watson the sloth: B-rave. Braaaave-   PA: Brave starts with B-   “B” is made salient-   PA: Brave-   PA:This boy is brave, he is not afraid to climb very high in this    tree to save his cat.-   PA:Someone who is brave will do things that are scary, dangerous, or    hard when they need to.-   Chime noise-   “Brave” changes to “Valiente”

PA: Brave quiere decir valiente. Cuando alguien es valiente no tienemiedo de hacer cosas peligrosas o difíciles.

-   “Valiente” changes to “Brave”-   PA: I'm not scared to do things that are scary or dangerous or hard    when I need to, I'm brave. Let's say: I'm not scared, I'm brave!-   Some form of looking brave.-   Script 3 (See FIG. 4)-   PA: Bounce-   Watson the sloth: B-ounce. B000uunce.-   PA: Bounce starts with B “B” is made salient-   PA: Bounce-   “Bounce” changes to “Bounced” when PA says bounced-   PA: Look at this girl bounce on the trampoline. She did this    yesterday. Yesterday, she bounced up and down on the trampoline.-   “Bounced” changes to “Bounces” when PA says Bounces-   PA: When something bounces, it moves up and down, again and again.-   Chime noise-   “Bounces” changes to “Rebotar”-   PA: Bounce quiere decir rebotar. Cuando algo rebota se mueve hacia    arriba y hacia abajo una y otra vez.-   PA: I'm bouncing. Let's act out bouncing. You move up and down again    and again.-   “Rebotar” changes to “Bouncing,” PA bounces.-   Script 4-   PA: Hunt-   Watson the Sloth: H-unt, huunt.-   PA: Hunt starts with H-   “H” is made salient-   PA: Hunt-   PA: This cat went outside to hunt. Yesterday, she went hunting for a    mouse to eat.-   “Hunt” changes to “Hunting” when PA says hunting-   PA: When a person or animal hunts, they go look for something to    catch and eat.-   “Hunting” changes to “Hunts”-   Chime noise-   “Hunts” changes to “Cazar”-   PA: Hunt quiere decir cazar. Cuando una persona o un animal caza,    salen en busca de algo que atrapar y comer.-   “Cazar” changes to “Hunt”-   PA: Pretend you are a cat and you are hunting for food. Say, I'm    hunting for food!-   PA does some hunting action.

Script 5

-   PA:Wsh-   Watson the Sloth: W-ish, wiiiish-   PA:Wish starts with W-   “W” is made salient-   PA:Wish-   PA:This boy made a wish on his birthday. He wished for a puppy.-   PA: When you wish for something, you think about something you want    to happen or come true.-   Chime sound-   “Wish” changes to “deseo”-   PA:Wish quiere decir deseo. Cuando haces un deseo piensas en algo    que quieres que ocurra o que se haga realidad.-   “Deseo” changes to “Wish”-   PA:Pretend it's your birthday and you are wishing for your favorite    toy. Say, I'm wishing for my favorite toy!-   PA does some wishing action

Script 6

-   PA: Silent-   Watson the sloth: S-ilent, Siiilent.-   PA:Silent starts with S-   “S” is made salient-   PA:Silent-   PA: This girl is telling us to be very quiet and not make noise. She    is telling us to be silent.-   PA: Someone who is silent is completely quiet and is making no sound    at all-   “Silent” changes to “Silencioso”-   PA: Silent quiere decir silencloso. Cuando alguien es silencioso    está completamente caliado y no está haciendo ningOn mid°.-   “Sliencioso” changes to “Silent”

PA: Pretend you are in a noisy room and you want everyone to be silent.Say, shhhhhh. Please be silent!

-   PA does a silencing action.

Script 7

-   PA: Thought-   Watson the Sloth: Th-ought, Thouuuught.-   PA: Thought starts with T-   “T” is made salient-   PA: Thought.-   PA: Look at this girl, she has thoughts about spaceships, stars, and    the moon.-   PA: A thought is an idea you have in your mind-   Chime sound-   “Thought” changes to “Pensamiento”-   PA:Thought quiere decir pensamiento. Un pensamiento es una idea que    tienes en to mente.-   “Pensamiento” changes to “Thought”-   PA: Pretend you are thinking. Touch your head and say, I had a    thought!-   PA does a thinking action.-   PA: Flap-   Watson the Sloth: F-lapped, fllaaappped-   PA: Flap starts with F-   “F” becomes salient-   PA: Flap-   PA: Look at this bird, yesterday he flapped his wings. He moved his    wings up and down very fast.-   “Flap” changes to “Flapped” when PA says flapped-   PA: If a bird flaps its wings, his wings move quickly up and down.-   “Flapped” changes to “Flaps” when PA says flaps-   Chime sound-   “Flaps” changes to “Aletear”-   PA: Rap quiere declr aletear. Cuando un pájaro aletea sus alas las    mueve rápidamente hada arriba y hada abajo.-   “Aletear” changes to “Flap”-   PA: Open your arms and pretend to flap your wings like a bird. Say    I'm flapping my wings!-   PA does some flapping actions

1. A computerized system of teaching that utilizes curriculum materialshaving encoded indicia, comprising: a computer comprising computerizedmemory and a processor executing imaging software and decoding softwarein said memory; a camera comprising an image reader assembly configuredto generate pixel data from the encoded indicia on a respectivecurriculum material; wherein the imaging software processes said pixeldata to generate digital pixel data and stores the digital pixel data insaid memory, wherein said processor executes said decoding software byreceiving said digital pixel data from said memory and generates a dataset from the encoded indicia; a transceiver in bi-directionalcommunication with the computer and a server on a network, said serverhaving access to augmented reality image data, said transceivertransmitting said data set to said server over the network; wherein saidserver is configured to receive said data set and transmit, to thecomputer, selected augmented reality image data that corresponds to thedata set, and wherein said computer uses the processor to show theaugmented reality image data on a display.
 2. The system according toclaim 1, further comprising a projector configured to display theaugmented reality image data in real world surroundings.
 3. The systemaccording to claim 1, wherein the display on the computer is configuredto show the augmented reality image data in synchronized video formatand audio format.
 4. The system according to claim 1, further comprisinga graphical user interface connected to the computer, said graphicaluser interface allowing a student to interact with the augmented realityimage data in real time communications over the network with the server.5. A system according to claim 1, wherein said encoded indicia is a QRcode and said data set transmitted to the server comprises at least aportion of information within the indicia.
 6. A system according toclaim 5, wherein said data set comprises an identification numbercorresponding to a respective curriculum material.
 7. A system accordingto claim 5, wherein said encoded indicia comprises vocabulary wordinformation in addition to said data set transmitted to said server. 8.A system according to claim 1, wherein the display on the computer isconfigured to show the augmented reality image data in synchronizedvideo format and audio format, wherein said audio format comprisesbi-lingual content.
 9. A system according to claim 8, wherein saidbi-lingual content comprises augmented reality instruction in astudent's first language and a second language.
 10. A system accordingto claim 9, further comprising a graphical user interface and voicetransmission software allowing a student to interact with saidbi-lingual content to practice a vocabulary word explained in thebi-lingual content.
 11. A system according to claim 1, wherein saidcurriculum material is a flash card associated with a vocabulary word,and the encoded indicia is positioned on the flash card.
 12. A systemaccording to claim 1, wherein said augmented reality image datacomprises video data configured to implement a pedagogical character onthe display.
 13. A system according to claim 1, wherein said imagingsoftware generating said digital pixel data stores said digital pixeldata in addressed frames of data, and said data set comprises at leastan addressed subset of said frames of data.
 14. A system according toclaim 1, wherein said computer is a mobile computer or a smart device.15. A computer program product for, the computer program productcomprising: a set of computer instructions stored on non-transitorycomputer readable media housed in a computer, said instructionsconfigured to be executed by a processor to implement a decoding processin regard to an image of an encoded indicia on a curriculum material,wherein said computer instructions comprise: an imaging module that (i)activates a camera in the computer to capture an image of the encodedindicia, (ii) generates a set of digital pixel data corresponding tosaid encoded indicia, and (iii) stores said digital pixel data inaddressed memory locations in the computer; a decoding module thatprocesses at least a portion of the digital pixel data to generate adata set corresponding to a respective portion of digital pixel datastored in memory; and a transmission module configured to execute a datatransmission via a network in communication with said computer, the datatransmission sending said data set to a remote server that identifiescorresponding augmented reality image data requested by the data set; asecond set of computer instructions stored on said non-transitorycomputer readable media and executable by said processor to (i) receivevia said network the augmented reality image data from said remoteserver and (ii) display said augmented reality image data on saidcomputer.
 16. A computer program product according to claim 15, furthercomprising camera software configured to establish a field of view forthe camera in the computer.
 17. A computer program product according toclaim 15, wherein said imaging module is configured to process eitherone dimensional or two dimensional encoded indicia.
 18. A computerprogram product according to claim 15, wherein said encoded indiciacomprises identification data that corresponds to a version of augmentedreality image data on a server that is tied to the curriculum material.19. A computer program product according to claim 18, wherein theaugmented reality image data comprises pedagogical agent image datatailored to the version.
 20. A computer program product according toclaim 18, further comprising a graphical user interface module allowinga student to interact with the augmented reality image data.